The disclosure relates generally to partial stochastic rounding that includes sticky and guard bits.
In general, machine learning and neural network applications require a method of rounding results using a random value to determine if a fractional part of an intermediate result should cause an increment (rounding up) or truncation (rounding down) of the final result. For instance, in a contemporary implementation, a method of rounding can determine a cost of a product to the nearest five cents to eliminate the use of pennies. Yet, when 10,000 products are sold at the cost of $9.98 cents, a seller will likely always receive the benefit of the rounding (e.g., $9.98 more often will round up to $10.00 than round down to $9.95). At present, the contemporary implementations of these applications fail to avoid a tendency for one side to always benefit when attempting to implement methods of rounding.